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PREFACE 


Work on this thesis was begun during the first eleven weeks of 195) 
while the writer was employed by the electronics design and mamfactur- 
ing firma of Gilfillan Brothers, Incorporated, located at Los Angeles, 
California. This employment was the industrial experience tour in the 
Ingineering Electronics Curriculum of the United States Naval bostgrad- 
uate School which requires the student to work as a junior engineer in 
the electronics company of his choice. Completion of the thesis was 
accomplished during the Spring of 195), at the United States Naval Post- 
graduate School. 

This investigation was done in cooperation with Mr. louis A. Ule, 

a member of the Theoretical Analysis Group in the Ingineering Department 
of Gilfillan Brothers, Incorporated. Its purpose was to round out the 
least squares smoothing and prediction theory embodied in Mr. Ule's 
intra-company memorandum of January 12, 195), entitled, "Polynomial 
Filters." While a considerable amount of material in the thesis is the 
result of joint discussion or discovery, the formlation of the weighting 
function (Chapter IJ, Section ) and the proof of the steady state filter 
resvonse in the absence of noise (Chapter II, Section 5) are abstracts 
of material written by hr. Ule. 

The writer wishes to express his appreciation to hr. Ule for his 
guidance, assistance, and original material. He is also indebted to 
Professors Abraham Sneingold and Robert Kahal of the United States Naval 
Postgraduate School Staff for their suggestions anu encourageneni while 


acting as thesis advisers and readers. 
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SYNOPSIS 

The sroothing and rrediction problem is one of extracting froma 
noisy signal the best estimate of some functional of the true signal at 
some past present, or future time. The historical develonment of the 
least squares solution to the problem is oriefly discussed alone with 
other backcround information relating to this process. 

A method of smoothing and rrediction is developed which is a dynamic 
form of least squares curve fitting. The truc signal is assumed to’ be 
composed of completely forelmovm functions of time cach multiplied by a 
constant coefficient. The filter Weighting function is chosen so that 
the filter output will be comosed of the same tunctions as the true 
signal except for a possible translation in time, differentiation, or 
integration. The squared error is time weichted before being mininised. 
While the derivation of the weighting function is inderendent of the type 
of noise, the choice of the squared error weichting function may depend 
upon the character of the noise. Wunctions which can be treated are found 
to be the solutions of the reduced equation of a linear differential 
equation with constant coefficients. The resultant filter weighting 
function derends upon the true signal, the rrediction time, the squared 
error weighting, and an inverse matrix wich is the solution of a set of 
integral equations. 

The resultant filters are constructed of lincar elements Which do not 
vary with time. Although no conditions for niysical realizability are 
inmosed, a high probability exists that the filters may be constructed by 
one of several methodse A step by step procecure for the fliter design is 
given together with specific examples of its use. A phenomenon atcin to 


resonance is observed for e:xmonential and polynomial functions. 








CHAPTER I DVMYRODUCTIOI! Tu ThE SMOOTIOIIG AND PREDICTION PROSLEM 


1. Description of the Problen. 

The smoothing problem is one of extracting true signals from noisy 
onese This problem has beon described by Bode and Shannon* and by Zadeh 
and Ragaszini 3%, It is usually assumed that a given perturbed signal, 
R(t), is the sum of a true signal, P(t), and a random disturbance or 


“~ 


noise, W(t)e 
R(t) = P(t) + N(t) (1) 


By performing appropriate operations on R(t), it is desired to obtain a 
result Q(t) which is the best approziimation of P(t). This snoothing 
process may be corbined with prediction to provide a continuous approzw- 
imation of P(t) at some past or future tine. In other words, given R(t) 
and a fissed preaiction tine 6, what is P(t + 8)? © may be positive, 
negative, or zero cepending upon whether prediction, time delay, or 
siootling alone is desired. Both sacothing and vrediction may be fix 
ther conbined with differentiation and interration so that in the ger 
eral case some functional of P(t + 6) is obtaincd. Commonly desirod 
functionals are of the form P(t + @), dP(t + @)/dt ana [P(t + 6) dt. 


The problen of filling the box marked "2?" is illustrated in Figure l. 










"Optimum" 
Transfer 
Operator 
? 


R(t) = P(t) + Nt) Q(t) = best approximation 


of functional of P(t + 8) 


Figure 1. The Smoothing and Prediction Problen, 








As a class, tne plysical cowrtcrjarts of the "opti: trausler 
opcrators are coumonly lion as seoctliing and predicting filters, cstie 
ators, or predictors. Since any one filter tm, not perform all of the 
aforesaid processeo, specilic names to indicate @imctly Wit cach filter 
does :aght be in order. Hem les of Such nawiies would be "reproduccr® to 
indicate a saoutling process witn no time lag or lead; "differentiating 
predictor" to indicate both dilferentiation and awothing with a fined 
time lead; and "integrating, delayer" to indicate poth integration and 
smoothing with a fined tine delay. 

The design of an optimum predictor involves tii questions. in the 
first placc, wiwt are tiie transidr charactcfistics for the "optim 
trausser operator? Secondly, how may tnese charactcristics be physically 
realized? Before the first question can be answered, a criterion of 
perirormance must be ciosen and a method of charactcrising; the signal and 
noise mist be found. By c.100sing a realistic critcrion, the ansver to 
both questions may be creatly simplified. ‘he prover characterization 
Of the signal aud noise will pert one to mice the optimum use of the 
@ priori information concerning tie signal and nvise. 

While the sroothin: and prediction Problen is usually thought of as 
one in cormmnication theor,, it dent well be generalized to cover such 
diverse fields as gunner,, economic prediction, weather forecasting and 


statistics. 


=. Performance Criteria. 
To attacia tue provle: mathematically, a criterion of vcerfornance or 
measure of fidelity sust be chyusen zor the optim predictor. This 


eritcrion shouid be realistic 18 order that the results have meaning. 











It should also be simple in order that the nmathetatical treatment may be 
economically possible. liormally nezative crrors are as undesirable as 
positive ones so that the chosen critcrion should be an even function of 
error. 

Grahan and Laie” have esmurined the criteria for the “optimum 
transient response in linear transfer systems called "duplicators," in 
which the output appro:anmately dunlicates the input They have concluded 
that the minizun integral of the tine-miltiplied absolute value of error 
criterion is deste 


1) 
I= va t | e(t)| dt = mninimn (2) 


Oo 


The results of this article do not appear to meet the requirements of 
the smoothing and prediction problem since the lack of a noise component 
in the input signal has been tacitly assumed. 

ror signals which consist of a numoer of discrete pieces of data, 
the probability that an error exists might be used as a measure of 
fidelity. it is meaningless in the case of continuous signals in tne 
presence of noise since the prooability that an crror will occur at any 
Given point is alveys one. 

If the error is designated as e(t) and the mean value of a quantity 
by a bar apove that quantity, a perforuance criterion for continuous 


signals mght eaploy the minimization of one of the following: 





Eo) 





en : ae : 
e (t) where n is a positive intecer 








probability that |e(t)| >a specified linit 


probability that 1iission is not accomplished 
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The mean-square crror criterion introduced by hharievitch™” has 


found wice usage. Tids criterion requires that: 


Tn 





1 





e@(t) = e“(t) dt = niniam (3) 


ete 
ab 

Fron equation (3) it is evident that the undesirability of an crror is 
weighted according to the square of its magnitude. The 1.Yrinciple atten 
tion is paid to making the very large errors as siail as possible with 
little regard to snall ecrrors. Within the time range of interest the 
weighting is independent of the tie when the error occurse 

There are several reasons why this criterion has found such wide 
acceptance. First of all, it is gencrally applicable and in reasonable 
accord with physical requirements. Ilumerous cases esist in which the 
undesirability of an error increases with its magnitude. The mean-square 
error criterion is reasonable when the error has a normaal or Gaussian 
distribution which is independent of time, a condition ofter encountcred 
in practicee In the second place, the mean-square crror method elim-~ 
nates mich of the mathematical comlerxity involved in the use of other 
criteria. A highly developed body of mathenatical Imovledge has been 
built around the idea of the mean-square value. In some cases, it is the 
only criterion which is amenable to mathemtical treatuent. Finally, 
this criterion leads to useful results. Information can often be ob- 
tained from titis netiiod even when it is not strictly acplicable. 

The mean-square critcrion does not alivays meet the requirements of 


= 
the system. Ragazzini and Zadeh“? have presented the case in which all 








errors larger than a certain limit are qyuall; vad, wile those less 
than the limit are oqually acceptable. This is the classic pumery 
problen best descrived by the atciom, "A miss is as good as a mile." 
Cases May occur in which the undesirability of an error dezends unon 
the tine at waich the error occurs. For q@wample, the undesirability 
of ameerror during @ transition from one ede of oneration to another 
may be quite different from that while operating in a given mode. 
Another case in which the mean-square error criterion would not apply 
involves the times when as imny very accurate p-reuictions as possible 


are desired and an occasional fross crror is of little consequencee 


3. IGstorical Development. 

The first successful attempt in this country to rut the noise 
problem on a mathematical basis was Norbert Wiener! 5 classic report 
for the National Defense Rescarch Council entitled, "The Extrapolation, 
Interpolation, and Smoothing oi Stationary Time Serics," which appeared 
in February 19)2. An earlicr, similer but not identical article by 
Ae ile Kolmogorore’ appeared in a Russion scientific journal in 191. 
Although both worl:s are cevotcd to a study of the optimun prediction 
problen through the use of statistical methods and support cach other, 
they are the result of independent investigations. 

The WiencrKoliogoroff theory is based on three assuzptions which 
limit its range of application. 
le The signal and noise nay be satisfactorily represented by stationary 
time series, that is, the statistical rrovertics of the signal and noise 


do not change titi tinee 








ee ‘The winimui mean-square error is a reasonable criterion of perform 
ance. The ensemble average is tal:en over all yossiole signal and noise 
functions, each weighted according. to its probability of occurance. 

3- The operation used for smoothing and prediction is linear so that 
it may be performed by a vhysically realizable filter. 

For these reasons the theory has been described as "Linear least square 
prediction and smootiung of stationary tine serics." If a given time 
series is stationary and the correlation fiuncticns of the signal and 
noise are Inowm, this theory nay be used to obtain the snecifications 
for an optirnni predictor. 

An early application of Wiener's work by PhiJlips and Vieiss“ 
extended the theory to quasi—stationary processes, that is, processes 
Which are stationary in the range of interceste The problem under 
investigation tas the snoothing of data obtained in the tracking of a 
gumery target. In this case, the signal and noise characteristics 
are considered invariant except when the target changes course. Then 
the error is large cnough so that the possibility of a hit is necli- 
gible and the results nced not be considered. 

Because of the forsiidable mathematics involved in Wiener's theory, 
llorman wane” Ee has written two papers with a view tovard facili- 
tating the computational procedure and surmarising the significant 
results. Bode and eeemonc have presented a simplitied derivation of 
the theory which does not resort to the use of integral equations or 
auto-correlation functions. Recent books on information theory, such 
as those by Goldaan ! and pel’, discuss the noise problen. 

Zadeh and Regecea have further e:ctended the tneory in two wayse 


The desired signal may also contain a nomrandom fimction of tine which 








is representable as a polynomial of decree not greater than n and about 
which no information other than nis lmowm. In addition, the imw sive 
resvonse of the predictor may vanish outside a svecificd tine interval 
0 S$ t -< T, whereas T is infinite in Wiener's thoorye 
dies : , 
A recent paper by Hauser” discusses the geometric aspects of the 


least squares smoothing process. 


le Time versus Frequency Domain Linear Filters. 

The most inportant prerequisite of any filter is that it be physi- 
cally realizable. Unless this condition is met, any "optimun" transfer 
operation is of academic interest only. For case of mthenatical 
treatment and practical realizability, it is usually desirable that the 
filter also be fixed and linear. Janes and Weiss! have indicated the 
charactcristics of the normal response of a lincar filter. The nornal 
response should be a linear function of the input so that the linear 
superposition of inputs and resultant responses will hold. It should 
depend only on the past and present valucs of the inpute In addition, 
this response snould be independent of the tine origin. This means 
that the values of circuit elements are fixed and independent of time. 

The classical approach to the problem of separating signals fron 
interference, whether due to noise or other signals, is to provice a 
filter of limited bandwidth as deter.iined by the stcady state frecuency 
components of the desired signal. The resonant tuned circuit for sim- 
soidal signals is a good example of tiiis teclmiyue.e. For signals ofa 
transient nature which have neither bandwidth nor frequency coimonents 
in the usual sense, this method may be clumsy. 


With the increased use of the operational mathematics, it has become 





conventional to utilize the Laplace transforn whien dealing with filters 


for transient signals. ‘The situation is represented by Figure 2. 


F(s) H(s) G(s) 


Figure 2. Frequency Domain Filtering. 


F(s) and G(s) are the Laplace transforms of the input and output 
Signals respectively. ‘The transfer function of the filter H(s) is 
defined as the ratio of the Laplace transfcoims of ay normal response 


and the input that produces ite It llows that 
G(s) = F(s) H(s) (1) 


The filtcring of signals which are functions of time can also be 
done directly in the time domain. The input signal is f(t) and the 
filter is compictely described by its weighting function W(t). Tris 
weighting function, often called the impulsive resvonse of tne filter, 
2S the normal response of the filter to a unit impulse aprlicd at zero 
time. The weighting function expresses the ccatent to wrici the distant 
past of tne Liput arfects ti-e present Miter res_onse and nence may be 
tnovught of as the mexery of the filter. The outout of the filter, g(t), 
is given by the convolution or superposition intcgral in terms of I(t), 


Wt), and a dumy variable of integrations. 


t 
g(t) = f(t) * W(t) = i f(t -0) w(t) at | (5) 
O 


Time domain filtering is illustrated in Figure 3. 














£(t) W(t) g(t) 


Figure 3. Time Domain Filtering. 


The following correspondence exists betiicen the time and frequency 


domain filters. 


Rs) =L | 2(+)] (6) 
n(s) = Z [ w(t) | (7) 


This duality between the time and frequency domains is further shown by 


the Gardner and pate? complex miltiplication Laplace transfor pairs. 


f(t) +t f(t) —_— F,(s) FY, (s) 


A technique utilisin; both the time and fre ,uency domains will be 
used later. It consists of finding the appropriate weiz:hting function, 
W(t), and then talcing its Laplace transfor to obtain Ii(s). In tis 
case the vrovertics of the Milter are wore readily; found in the tine 
domaine The sritch to the frequency domain is employed so that con- 


ventional methods of network synthesis may be used. 


5. lhmerical Least Squares Curve Fitting. 

Related to the discussion of the seccnd section of this chapter is 
the static case of least syuares curve fitting. scaiiorowgh” has dis= 
cussed both the basis for the method and the steps necessary to carry 


out the mumerical computation. It is basec on the Normal Law of Error, 


LO 








the Principie of least Squares, and the Law of Eriror for Residuals. 
Consider the function, Pb, which is the sum of séveral functions, f(x) " 


f(x)s ena f(x). 
P me by £, (x) + »b, f(x) a £ (x) (8) 


Pairs of observed values, 2 Pi), (xc. P,), ae (3_, P are ob= 
tainea irom a sect of m measurements of ecual precision. It is desired 
to find the values of the cocfricicnts, Dy» Des ae ae by wach that a 
graph of P will come as near as possiole to cach of the mn points. A 
set of m residual equations is obtained by substituting each of the 


pairs of valucs in turn into cquation (8). 


y= bf) (%)) + Defi) + --- + DF CH) - PY (9) 
Vo = bj i) (%) + b-f,(%) + »-- + DEH) - Po (10) 
mo biG) + biog) + - 8. + BEG) - B, (11) — 


mie best valucs of the unlm@iowm cociiic#ents are Goose riiich make 


the sum of tne syuares of the residuals a 1.donimr, 


2 eo —_. a 
“ai + Le, + e ¢ e@ - a — i(b, ; Dos eo 8 @ db.) HE g HL 00a] (12) 


TO Malice ewuatiion (12) a Masciraii or minimm, the partial derivatives of 


f(b), bo eave bd) with respect to each of the coefficients snail be 
gero.e The values of the residuals, Vj Who > + + Me given in equations 


(9) through (11) are substituted into equation (12); the partial differ 














entiations are performed with res sect to by: Ds wm « » D § amc The n 

Z n 
resulting equations are equated to zero. Aftcr dividing through by 2, 
a set of simnltancous eyuations, called norval equations, are obtained. 


They may be solved by ordinary alyebrae 


£,(x,) fe £ 66) td f(x) +++. +d) - Ps 


cA. b ‘< . ee e© @ ok _ e ee e 
+ £,(e,)[d,£,(x,) + b,£,,) + tb ft (x,) =P | + 
t ls eb s Dt ; a es = 
P i! my 1 1%) a 2 ol Qn 7 2 os w J : (13) 


£04) fest (=) + d,£,0x,) _— bf Cy) - P| 


+ f(x.) bf (x,) + bf ,,Cs,) at b f(s) - P,, ee 


i > P| = 0 (11) 


+ 


mee abs te) + bof5Ge) 


+ 
e 

e 

e 
+ 
Cc 

= 

Lair) 
~~ 
» 


bE) 7 P, | 
oe + bt G,) - a pe 6 
-. df (x) - ‘, | = 0 (15) 


fio) e264) + bf G4) +a. 
+ f x,) b, £) (x) + b,2,(x,) 


+ £ Gs) b, f(x) ‘ b,f,(5) 


+ 
e 


= a 
e 


If each unlmovwn coefficient is to be uniquely determined, the 
number of indevendent scts of observations ust be eygual to or creater 
than the nummer of coeificients. By increasing the nunber of ovscr 
Mat2ons, a better fit is obtained. 

When measurenents are of unequal precisicn, the least square nethod 


is exctended by having the sun of the weighted squares of the residuals a 


Innimne 

- 2 é °C 2 

Weve = WY, + WyVn + oe + wiv, = ninimn (16) 
j=l 





The weights, wy; Wo, ¢ « + Why depend upon the frecision with wiich each 
pair of measurements is made, the mere precise measurements receiving 


creater weights. 


6. The Finite Fouricr Serics as a Minimum ‘liean—Square Appro:imtion. 













As an introduction to a technique which rill be utilised later, 
the least square property of the finite Fourber series will be show. 
Let a function, f(t), which is piccewise continuous and defined from 


c 4 t S$ c + 27, be approximated by thes Damctionyw(t). 


g(t) = ap + a, cost + by sint + ao cos Ot + bo sin 2t 


# 2 . . e+ coe nt  b, ein nt (17) 
n 

s(t) = i a cos ltt + b. sin kt (18) 
1=0 


‘It is desired to choose a. and b,. such that the mean-squcre error, e(t), 


over the interval for which f(t) is cefined is a minimm 


cme 2/7 
ett) = 2 / ES = c(t) | Ode eaten (19) 
277 
Cc 


For a minimum, the partial derivatives of e(t) with resnect to a,. and 
b,. should be zero. The valuc of ¢(t) given by cauation (18) is sub= 
stituted into couation (19). The partial derivative with resrect to 
ay. my be talsen under the integral sien and the resulting expression 


equated to zero. 


+ 277 


n n 
21f(t) = (a, cos It + b,. sin lt)] | - cospjtice =O (20) 
= j=O 
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n 
—.. a,.cogs ft cogs @ % b. gale ertU cae wv t 
77 ki I: 


Cc lem OQ == 0 1 c+ 27 s 
a. C(t) cos jt at (21) 
7 
Cc 320 ‘ 


To evaluate the leN side of e uation (21), certain orthoronal 


mropertitcs of thc Sfie cd cOUime SEoRicl bt Peco lcd. 








c+ 077 
eS nx cose da = 0 forn # n (22) 
C 
c+ 277 
cog@ rk cobux dk& = FF for n =n (23) 
c+ 277 
Sin nc com tec adn St O forall n @md no (2h) 


The only contrivution to the left side of equation (21) occurs when k 


- ‘and j are equal. Fquation (2) may now be written as 
c+ 277 


n n 
3 a = *L | f(4) cos lt @ (25) 
z=OvVe 


> k=0 


ms a result of the rearrangement of ccuation (20) to cet equation (21), 





there is an equality between terms which arise fron substituting the 


sar 





value of k on each side of equation (25). Hence the general 


esmression for 2. is 


» 


c+ OF 


a, = = f(t) cos Ist dt Were lo Ct, . ..n G26) 
4 C 


( Mis is recocnized as the gencral expression for the cocfficicnts of 

. the cosine terms in the Fourier scries @mansion. A similar process 
Seetaliin: the partial derivative with resnect to om wall yield tie 
@eneral excression for the coefficients of the sine terms in the Fourler 


serics cz:mansion; 











ihe r@mairimg aquectlun Yo be selti@l De whites Ule vedas oo | 
and b,. just oDUazned rommoset a Tlagtie, eiliiucem, cr «@nddlc pobyt. Fits 
is accomolisned by findiy; the values of the three scopy pertlal deme 


avives neyrcserttd@ig 4B, 2nd Cc alu Be +t (o., by). five Uitte 
de be 


@abiations Wer gain be carried out gad@r the Sygaeral Gicn. 


c# 2 
= F200 (au) 4/ y. ) cos It Cooeit Be = 1 (2575) 
eae ‘ 4 
wie =Q j=0 
+ 
oO dt = 0 (<6) 
Cc 
+ cM, n 


sin It sin jt dt = 1 (29) 





The conditions for a relative minima of a function of tro variables are 









mo - « <0 (30) 


= c > 0 (31) 


Poet concitions are mct in this case. 





Thus if f(t) be niccevise continuous for c < t= 


c + 27, then 
mee cOcificicnts of the vartial sum of the Fouricr scrics of f(t) are 
peecrscl; those anone all cocfficients of the function, ¢(t), wich 
render the mean-square error a Lani. 

The foregoing vroof was succested by a nroblen in 'i1debrandt3, 
Both Guillesin? and Kaplent© prove the same result using different 


approaches. 


i 








CHAPTER II WW! EXPEMGIO! OF LUAST SQUARES PREDICTION TMBORY 


1. Basic Agetert one. 
fhe discusfion of tis clapter will be b@Bage ueon, Me p~eneral 


Problem as stated in Ceapter I. The reccivee, observed, or Meeasined 















Serra, R(t), witich if the inset to the filter, cortei@ts of a true 
Saen-l, P(t), and noise, M(t). The true Signal, P(t), will be ascmed 
ho consist of n cosréletcl: foremotm finctions of time, £5 (%) P fo(t)s 
_. f(t), each :mltiAlicd by 2 constant cocfficient, 249 Agr 2 se 


ay Mitich is not a finction of times 


n 
P(t) = sil ai, fy (t) - 
h=1 


_ } azt ; : ; : 
puen functions as c 1” and sin a,v are not considered since they are in 
G 





effect undctertined finctions. 

From a forelmowledce of the form of P(t) and ow operations on 
mi ),.We attempt to obtain a closer estimate of P(t), exlhed Q(t), such 
that Q(t) will differ from R(t) by the least amount in some manner or 
Other. The “best" estincte Q(t), ill be chosen to have the same fona 


@e P(t), that is 


n 

at) = 5 ng. tt) Gh) 
i=l 

The functions ¢).(t), in the emression for w(t) ate clos@l:;; related to 


the functions, f},(t), in the expression for P(t) and in many cases may 


be identical. The exact restrictions imposed upon the functions nalting 
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up P(t) and Q(t) will be discissed # the ze:t sectitn. No asmumtion 
will be made abovt tue type of noise, W(t). It willeme assumed tat a 
Peameneble criterion of 1 crifemmance is onc ia walich tie time weigiiad 


Savcrea aur Ls frinbrilsed.as 7 lluserobed inugeetten ti now of Tie 










Ewepter. Tiis :amizuscation £5 breughituabogs br Byoonm@e in whtich tire 


uncetcriined constants, bj, bo, « . . Bb, are celectcd. 


n? 


In or@er to simplifyY tRe plysteal recligatmes of tie filters, tie 





Mel be assumed to be constructed of linear cle@nente which do not ver; 
Maul tie. ‘Tite limited scope of titis paper Will not parmmit a thorough 
fieeton Ofwa broader class of filwens Wich May we constructed by 
Putting the sise of the filter elements to vary Witla tines 

In general, we assume the finction, P(t), to be comlctely fore- 
me. There are ex:ccevtipas, however, Which are agennble to tremtment 
if there is some decree of uncertainty as tc the fora of P(t). 
This occurs when the Amction is untmovn in the sense that it is one 

Of several imorm possible fimctions. Tne filter may oc designed to take 


mo. oll the Imovn possivilitics. We an eagles suppose that P(t) 





~ 


is snowm to be citner aj, ao of, or a5 e-%, Ten the actual Sipnel ror 


mich the filter is designed might be 
Q(t) = by + doe’ + bz ett (35) 


fee price paid for tris uncertainty; as to the emect function present is 
Seencreasc in the noise outout of the filter. This is so because tne 
@melusion of additional functions will in feneral require the filter to 
have a freater frequency resnonse than that required for the one true 


Signal alone. The increase in filter noise ovtmet limits the e:ctent to 
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Whick thas metvod can be cerried. sAvetHcr case is that of the undeter= 
Peed fus:cticn a sh Ga. + an). Ac it stond&, it ca@moet be trcamed. 
Mowever, Db ertling it adm Wleweqtivicnt fay by Sin We. ba acs Ft, 

G 


a6 tee be censidercd. In this caee tlerc is ho docs of gememallity ner 





















increase in the no“se outhut. 


@e Anplicable Functions. 
ihe only assurmtion vwitich will be madc about the tync of functions 

comprising Q(t) is that they for: a closed set under a linear trano- 
Mee2orn in time, that is, no new functions Will be cencrated. This 
meeeriction Yesults from the fact that, for Sa@mlicit,, tre filters are 
moidercd to be invariant with time. In going fron 2 Static case of 
Scot soueres curve fMiting to the dmanicitwation, no timze reference 
Beeorovided the filtcr. Given a sinusoidal type wave that has been 

ing on for an indefinitely long time, one cannot say whether the wave 

ha sinc or a cosine unless seme reference time 1s cfosen. Wowever, 
fo metter wnat refcrence is cuosen, the wave can alivays be described as 
ge linear combination of cos wt and sin wt. To uncerstand what this 
sriction iimlics, consider the fimctions b, sin wt and bo t® under a 


Peensi2tion in tine of duration ce 


by sinw(t +c) = bs sinit + by cos wt (36) 


2 


= 


bs (t + c)¢ = be + be tT = bo 42 (37) 


In both cases, new functions resulted from the translations in tine. 


Mee if the c:oressions on the ri¢lit sices of equations (36) and (37) 
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underro e further translation in time, the translctions will nerely 
change the coefficients without penerating amr new functions. ‘Tims 
the richt sides satisfy the requirement for fimctions which male up 
Q(t). It may also be observed that functions of the fora by teh and 


i) e , . ® 
ae 3 con never satisfy titis restriction. 


by + Do t + by “8 
The nesct question requiring an answer is, "What functions remain 
o closéd s@t under @ linear transformavion in tame?" As an illustration, 


consider the case in which there are tio functions involved. 
| g(t) = by £3 (t) + bo Co(t) (38) 
Tie lineax transforzation may be written in matrix form 


63 (t + At) A B Gy (t) 
= (39) 
Eo(t + At) C 6D | Bolt) 


But A, B, C,; and D mst be chosen so that when At equals zero 


(t + 0) L OF | cy (t) 
= (1,0) 
Bolt + O) e) al Eo (t) 


Mis condition is satisfred if 


g(t + at) 1+ ae best £5 (t) 


gp (t + At) c At 1+¢ d At g.,(t) 


meere a, bs c, amid are arbitrary constants. Subtract the matrix 
6 (+) 
(4) from both sides of the matrix equation (11) and divide by Ate 
coled 
"2 


1, 

















si(t $At) = g(t) 
a v | |e, (t) 


a 
At 
= (4.2) 
Sei C d| |c.(t) 
At 
Since this mist hold for all At, lect At anproach the limit zero. 
The resulting derivative is denoted by a prine. 
ge, '(t) ab {e,(t) 
la ‘ (43) 
g,'(t) c 6d | | e,(t) 
wow talzing the second derivatives with respect to tine, 
g,''(t) a by} ] cyt) 
= (Lt) 
co''(t) Cc d g.'(t) 


The matrix equations (43) and (lj) may now be used to find sintlar 
second order differential cquations in 6, (+) and Bolt). These equations 


taze the form 
git(t) + Cy e'(t) + Co g(t) = 0 where C, and Cy are constants (15) 


By using similar methods, it is evident that if Q(t) consists ofn 
functions, these functions must be solutions of the reduced equation of 
ann th order linear differential ecuation with constant coefficients. 
mis restriction is not as serious as it might first appecr since a tide 
Varicty of usefrl fimctions belong in tois ¢crour. Included are emo 
nentials, polynomials, sinusoids, and functions formed by addins or 


multiplying these types together. While the functions, f(t), that forn 
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the true signal, P(t), may be considcred to bé atfy one Solution of the 
differential equation such as a, sin wt, Q(t) must consist of the corre 
snonding complete set of functions, b) sin tt + bo cos wi. Sirnlerly 
if f(t) has a polynomicl term such as ao t*, e(t) must contain the corre 


sponding complete polynomial, b3 +b), tt be t*. 


3- Time Weighted Least Squcres Curve Fitting. 

The process Witich will oe used to obtain a tine weicphted least 
squares fit is illustrated in Figure h. The ton sl:etch shows the noisy 
sipnal, R(t), and the best estimte of the true sirnal, Q(t). The 
difference between the two sipnals is the residval crror, R(t) - Q(t). 
In the middle of the figure is the square of tlis residual error, 

[R(e) ~ att)]* In order to allow some latitude in the construction of 
the filter, this squared error is weighted over the past in some arbi- 
trary manner by irmitiplying the square of the residual error by a 
M@eaciiting function of time, M(t). The nesilt of the multiplication 

is to produce a tine weighted squared residual errcr curve as showvm 

by the crosshatched area of the bottom diasram. Tre area underncath 


this curve is given by 


1@) 
. 4 R(t) = Q(t) - M(t) dt (1:6) 
[fae ~ ee) 


meas DYeciscel tris crosshatched ames which vill be minimised. 

The squared error weightinc functions M(t), may be thought of as 
cme filter's memoiy of the squared error. the rathenatical restriction 
on M(t) is that it be chosen so that the erea intecral is firite. There 


7S, however, another nractical restriction on I(t). Since one must wait 
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util the smothing timc, T, has been completed or until the filter has 
had tine to settle before it provides an "optimum" output, the menory 

of the filtor cannot extend too far into the nast. M(t) may be a com 
promise between a long smoothing time which makes use of more information 
about the signal and a short tine which would provide a quicker decision 
as to the nature of the signal. 

Probably the three most important forms of M(t) are the rectangular, 
exponential, and damped sinusoidal types shovm in Figure 5. In the 
rectangular type, the filter weights all saouared errors from a certain 
fixed time in the past, T, up to the present tine cqually. This corre- 


sponds to the weighting discussed by Zadeh and Ragagzini?? in which 
Mt) = ul(-t) = u(-t-T) where u(t) is the unit step fimction (1,7) 


The abrupt chanve in weighting results in filters which contain a delay 
line type clement. Another useful type filter is one which forgets what 


has gone on in the past esmonentially. Exponential weighting is defined 


by 
u(t) = of/T y(-t) (1:8) 


Tis type of memory is very promising in that it corresponds to the 
natural canning found in most physical systems tith dissinative elements, 
and hence holds forth the hone that the filters may be realized with 


lumped constant elements. ie third type of weighting defined by 
u(t) = eT (cos wt = sin wt) u(-t) (19) 


is an attempt to approximate the rectangular filter. The output of this 


filter would tend to have a transient damped oscillation and also holds 
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forth the hope of realisation with lumped constant elements. Other 
types of weighting which might find application under certain circun- 


stances include t 7% 4° et/T, and ot/T sin wt. 


le Formulation of the Weighting Function. 
If the expression for Q(t) given in equation (3) is substituted 
into equation (6), the expression for the crosshatched area in Figure 


l becomes 


oO n - 
2 
A = / Eo -) >», e.() u(t) at (50) 
= CO k=], ” 


the conditions for 4 mininum are that the partial derivetives of the 
expression for the area with respect to the coefficients b> Dos eo 8 


& 


by equal zero. This results in a set of n simultaneous linear equationse 


— 0 | where k = l; C3 e e@ e fl (51) 


In order to carry out these differentiations under the intesral sign; a 
form of Leibnitz's rule similar to that given by Keplan?! is now stated. 

let f(x, t) be continuous and have a continuous derivative df/ex in 
a domain of the xt plane which includes the rectangle a < x S b, 


tS tS 25 Then fora< x <b 


to to 
d ’ = of 
— f(x, t) dt = — (x, t) dt (52) 
ax ox u 


The partial derivative with respect to the general coefficient, bes will 


be talzen under the interral sign. 











O rn 

OA .. 2 

a -/ sp [ace sf b,. a.tt) M(t) dt (53) 
Oo 


=00 kal 


nN nN 
3A 
|  — af 14(t) Day(s fate f ) ao] at (5h:) 


—09 j=l ]: sa] 











Me change of subscript from Fk to j Wes made to indicate that the 
Summations are distinct. Sctting the partial derivative ecual to zero 


and rearranging the resulting equation 


\o) n e) rg 
>> 5(t) W(t) at = IS by. 6y.(t) > 65 (4) M(t) at (55) 
Vc 


j=l -Co “k=l j'=j 





If two new symbols are defined as follows, 


Oo 
ie iu R(t) g(t) M(t) at (56) 
- OO a 
Bu, = Bs =/ g(t) g(t) M(t) at (57) 
-~ CO 


then Ba can be evaluated since gy (t), C3 (t), ore J) ore all imom, 
and the set of n simltaneous equations defined by equaticn (55) take 


the simpler forn 


Bea 5 yan 2) 


jal eel jimj 


In matrix form these equations become 


= (59) 

















Bet Cj; be the it!vense matriss of Both thed if 


Cry Ci9 eo oe @ Can Biy Bio a ee Bin i 0 ee @ @) 
Coy Cao eo e e Cn, Boy Bon ee e® @ Bo, O lL e ® 8) 
= (60) 


® e e e ® . e e e e e e e e ® ® e e @ e e e e e e e e e 


Bede sos Copia Sn oe = a ie See. 


Premiltiplying both sides of equation (59) by the G. mtrix 


Hi fo eee GH tT by 
C5 C5 ® e e Cn, Ly bo 

_ (61) 
Ona Cao aa Con qn on 


Sa, in the form of 2 summtion, 
n 
.: - On ti (2) 
i=l 


Substituting from ecuation (62) into equation (3h) 


n pal 
a) = > Yh ay a(t) (63) 


kel i=l 
TO avoid confusion with the actual variable, t, the duwy veriable of 
integration in equation (56) is changed from t to x. The resulting 


valuc of I; is substitvted into’ equation (63). 


n ral oO 
at) =) > Fi R(x) g(x) ule) ax | O14 ey-(t) (61) 
_, 


ke] i=l 


Rearranging equation (6) so that all terms are uncer the integral sign, 


‘Oo n n 
Q(t) -/ ial 3 ress) gs (cx) Q-5 gy-(t) E(x) dbx (65) 


—OO }2]l i=l 
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Tet Cy; be the inverse matric: of Bytes that 16 


Cail) Ci9 eo ee @ Cin By Byo es we Bin ag 0 o = Ss 0 
Co] Can ee ee e Cn, Bor Bon . = 6 Bo, 0 x e ee «6 0 
= (€0) 


Cry Ono + © © Gon] | Boa Bho « + + Bon o.0, ££. 2 


Premitiplying both sides of equation (59) by the Q+~ mtrix 


OH, fo ee GH I by 
fe, G- eo «© « G I b 
el oe on as C 
_ (61) 
Or Cro 2 7 alia Con tn Os 


Sr, in the form of 2 swammtion, 


n 
bh. = » Gey dy (62) 


iml 


Substituting from eauation (62) into equation (3) 


n n 
Q(t) = r \ Is Gs £), (+) (63) 


kel i=l 
To avoid confusion with the actual variable, t, the dwry variable of 
integration in equation (56) is changed from t to x. The resulting 


value of Ij is substituted into’ equation (63). 


n n a 
Q(t) = ya f R(x) 63 (x) M(x) da} Cy g).(%) (6),) 
inl is —— 


Rearranging cquation (6) so that all terms are un’er the interral sign, 
‘oO on n 
Q(t) = va a i. R(x) g3x) G3 oy, (t) MGxc) d& (65) 
~“OO kel i=1 
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Consider that Q(t) is the output of a filter with an input, R(t). 
Referring to Figure lh, it is seen that Q(0) is the best ostimate of 
the true signal at the present time. If t is positive, the valuc of 
Q(t) is an estimate of a future value of the true signal. Thus the 
quantity t is the prediction tine. For the fixed filters under con- 
sideration, let t be a fixed prediction time cesignated by the symbol 
6. Bguation (65) may now be interpreted as a convolution integral of 


the form shovm by equation (66). 


Oo 
Q(6) = [ w(@, O ~ x) R(x) dx (66) 


A comparison of equations (65) and (G66) indicates that the weighting 


function of the filter mst be 


n n 
WM O-2) =) DY | eile) Ga gi (8) Me) (67) 


k=l i=1 


baling the substitution 
t= Ox (68) 


the weighting function of the filter becomes 


gl n 
WO t)= > YS) eet) Gg 6,6) Met) ult) (69). 


k=l isl 


where u(t) is the unit step function. In matrix form this becomes 


wie, t) = 


fey od ee ent-t) Ce uae 


g, (6) | M(t) u(t) 


(70) 


eal C0 eee one g, (8) 
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Since most network synthesis is done in the frequency domain, it 
is convenient to represent the filter by its comlex frequency transfer 


function, H(s), the Laplace transform of the weichting function. 


oO 
H(s) -[ wie, t) on at (71) 
@) 


To obtain smooth derivatives or integrals, H(s) is either miltiplied or 
divided by the variable se 


5e Steady State Filter Response in the Absence of Noise. 

If the filter weighting ftmctions found in the preceeding section 
are valid, a proper input to the filter in the absence of noise should 
produce a steady state output with no error. Since the noise, N(t), is 
zero, the input, R(t), is equal to the true signal, P(t). From equations 


(1) and (33), the input may be expressed as 


n 


R(t) = a. ay f(t) (72) 


h=1 
The filter weighting function is piven by equation (69) as 


n nh 
WO t= YY 2 cyt) o.5 oi-(0) ult) w(t) (73) 


kml im) 
From equation (5), the steady state output of the filter is 
'e-@) Oo 
Q(t) = We) R(t -%) ay = W(~T) R(t +7) at (74) 
Oo Oo 


Substituting the expressions for the input and weighting function into 


the equation for the steady state output, one obtains 


n n n 
Qt) = g3(T) G3 e,(0) Mr) a, f(t + T)|at (75) 


— kel isl hs] 




















n 


Assume that f(t + U) may be expressed as 
hel 


7" f,(t +t) = ds "ha Dyip(t) ep(%) 


hal ht'=h pl 
If — ie is substituted into equation (75), then 


(76) 


Q(t) +> t % » C+ &,(8) a, apt 6,(0) (2) MZ) - 


kel hel i=l pzl 
From equation (57) the integral By, may be defined as 


oO 
Bip = Df B4(C) &(%) M(t) ae 
— oO 


This will reduce equation (77) to the forn 


Qt) = - s : a Bin Cig 6,(@) ay Dyp(t) 


kelh=l 1=1 p=1 
The relation between Bip and Qy is 


Bip Ai = 1 when p = |: 
= 0 when p # k 


Equation (79) may now be reduced to 


Q(t) = 3 s- gy(8) ap Dy {t) 


kel hxwl 
By — with equation ae 


es Diy (t) @,(@) = $ f(t + @) 


kml hel h=1 
so that Q(t) can be expressed as 


n 
Q(t) = 3 apn fy(t + 9) 


hal 
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(78) 


(79) 


(80) 


(81) 


(82) 


(83) 














This expression may now be cormared with cquation (72) to give 


Q(t) = R(t + 6) : (8h) 












Thus the steady state outout of the filter equals the filter input 
displaced by the amount of the prediction tinc, 6. 

The only assumntion made in tlzis section was that defined by 
equation (76). It leads to a set of equations very similar to the 
matrix set given in equation (39). Tis asswmtion corresponds to 
the requirement that the outyovt of the filtcr have the same form as 
the input excent for a nossible translation in time. One may again 
Bekelude that the functions wnich are anenabdle to treatment are the 
so. lutions of the reduced equation of an n th order linear differential 


equation with: constant coefficients. 
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CHAPTOR IIT REALIZATION OF SMOOTIHTIG AND PREDICTING FILTERS 


1. Methods Available for Construction. 

the formmlation of the weighting function of Chapter IT did not 
insure that every filter was physically realisable using luwmed constant 
elements. It did, however, provide such a vide varicty of squared error 
memory functions, H(t), thet a high probability exists that the filters 
may be constructed by one of several methods. 

When a rectangular error memory function is used, it is often 
convenient to synthesize the transfer functions with operational ampli-~ 
fiers such as those used in analog corrutcrs. This is especially true 
if the signal under consideration is composed of low frequency cor 
ponents only. Since the rectangular error memory function leads to a 
delay line element, the operational amplifiers have the advantage of 
providing isolation and a means of suming the undelayed and delayed 
Signals. While conventional celay lines could be used for shorter 
smoothing times, it would probably be necessary to resort to a contin 
uous tape or drum arrangement such as tHe one shovm in Figure 6 for the 
longer smoothing times. The actual synthesis of these filters will not 
be discussed in detail since the needed information may be found in 
books on analor eoummcess such as the one by Korn and Korn@9, 

If an exponential or damped sinusoidal error memory function is 
used, it is oftcn possible to synthesize the filter as a two terminal 
driving point impedance, Z(s). The inwt to the filter is a current 
which could be supplied from a nentode while the output is the voltage 


across the terminals. A driving voint immedance must be a positive real 


xe 

















Rotating Drun or 


Continuoys Tape 


ee 





Figure 6. !enetic Tane or Drum Delay Element. 


function. The well Imovm conditions for a rational fXinction to be a 
positive real function are 

Mes) is real vhen s is real. 

Z(S) is analytic in the richt half plane 

Poles on the boundary are simmle with positive; real residues. 

Re lacse)] > O vhen Re >| ZrO 

If the transfer function meets these conditions, the synthesis of the 
filters as driving point immedances may often be done by insvection, 
and can always be acco:mlished by the methods of either Brune! or Bott 
and Duffin?? 19, 

A second method for synthesizing the transfer function is available 
when the exponential or dained sinusoidal crror nemory fimction is used. 
The transfer Rees. H(s); maxy be considered to be the ratio of the 
Open circuit outout voltage of a tro-terminal vnair network to the input 


Z 


voltacse. Kal.ql 5 has shevm the conditions for realizing the transfer 
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function to within a constant factor using a finite number of lumed 
constant, linear elements. For the symmetrical lattice network these 
conditions are 

H(s) is analytic in the right half s plane. 

Poles on the boundary are simile with residues having a zero real part. 
It is evident that a wider class of functions may be treated by this 
method than in the case of driving noint imedances. tbowever, there is 
a practical problem which arises from the fact that no cormon ground 
exists between the input and output terminals. This difficulty will not 
be present if the synthesis is carried out as a ladder netvrork. Kahal15 
has shown that one additional condition besides those for the symmetrical 
lattice must be satisfied to realize the transfer function to within a 
constant factor as a ladder network. This condition is that H(s) mst 
have no zeros in the right half plane. A major problem in the actual 
synthesis of the filters is usually the factorization of the transfer 
function. Tio methods for carrying out the actual synthesis of the 


filter are those of Darlington’, and Weinberg2?s 28, 


2. Filter Design Procedure. 

Before proceeding with examles of svecific filters, it.would be 
well to list the general steps to be followed in the design of any 
filter. Teneiily the function(s) to be smoothed, the prediction time, 
and the functional cesired are determined by the requirements of the 
problen. 


Step 1. From the know finction(s) to be smoothed, find the comlete 


set of functions which form a closed set under a translation in time. 


3h 














If the smocthed integral of a polymonial is desired, the filter must be 
designed for the next higher order polynomial. Conversely, the smoothed 
derivative of a polynomial will be designed for the nest lower order 
polynonial. 

Step 2. Choose a squared error memory function, M(t). 

Step 3. Find the elements of the B 3}, matrix in accordance with equation 
(57). 

Step }. Find the Q4 matrix which is the inverse of the By, matrix. 
Methods for obtaining the inverse matrix are given in the mathenatics 
texts by Pipes? and Guillenin', 

Step 5. Find the numerical values of the functions of step 1 at the 
prediction tine, ©. 

Step 6. Determine the filter weighting fimction as given by equation 
(70). 

Step 7. Taze the Laplace transfera of the weighting function. Mrltiply 
the result by s if a derivative is desired and divide it by s if an 
integral is desired. 

Step 8. Use the complex frequency transfer function found in step 7 as 


the basis for the gmthesis of the netvor. 


3. Smoothing and Prediction of an Esponential Function. 
The only functions which are soluticns of the first order linear 


differential equation with constant coefficients 
aoe Co fe 0 where C,; is a constant (85) 


are of the type aj cht, the steps of the preceeding section will be 
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followed to find filters which both smooth and tredict for this function. 
For the first case 1t 15 assumed that rectanguler weighting of the 

squared error is to be used to smooth the noisy function az et, the 

best estimate of the function itself is desired with a prediction time 


of 8 seconds and a snoothing time of T seconds. 








Sten l. g, = et (86) 
Step 2. L(t) u(—t) — u(-t~T) (87) 
0 
Step 3. By of eat at = = le real (88) 
-T 
om 
Sten h. 471=—-—-—_ (89) 
rm] — | 
Step 5. 6, (6) = e™® (90) 
: om gle - t) 
Step 6. w(t) = ——. [ u(t) -u(t = 7) | (91) 
md em! «Ts 
Step 7. H(s) = ee. | | dee Se... (92) 








La got (s + n) 


Step 8. One possidle method of synthesizing the transfer fimction is 
shorm in Figure 7. Operational amplifier elements are indicated by 
triangles. Actual values of the clements are not shovm since they denend 
upon 9, T, the type of operational amplifier used and the anticipated 
signal swing. Either the delay elenent or its associated operational 
amplifier must provide a negative output with respect to the input in 
order that the final summing amplifier may perform the subtraction 


indicated in equation (92). 
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oLen 1. 
Step 2. 
Step 3. 
Step lh. 
Steno 5. 
Bren O-. 


Step 7. 


Figure 7. 


Z(s) 


a 
> 


the exception that exponential weighting will be used. 


gy, = om 


L(t) = et/T u(—t) 
° 
pat at/T oye 1 
an + 1/T 





Bi, = 
—- 60 


Cq7 = n+ YT 
61 (6) = &® 


w(t) = elon + ao] e+ 1/7) t ut) 


er © (2n  1/T) 
s¢n2¢ VT 
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Exponential Predictor with Rectangular Weighting 


E out 


The second case to be LlLlustrated will be similar to the first with 


(93) 


(9);) 


(95) 


(96) 


(97) 


(98) 


(99) 

















Sten 8. The synthesis of the filter as a driving point impedance is 


illustrated in Figure &. 





_———— 
I in 
E out 
C= ~ —_—-~ R= (zn? + 1) o™ 
(on + 1/T) e? mt $1 


Figure &. Exponential Predictor with Exponential Weighting 


It may be noted that when m is negative, the values of T whicn will sive 
a realizable networi: are restricted. This has physical significance in 
that it is necessary for the noise output to decay at a faster rate than 


the signal for a usable output. 


lh. Smoothing of Sinusoidal Functions. 
The functions a; sin wt, a5 cos wt, and the linear combination of 
the tyvo are solutions of the second order linear differential equation 


with constant coefficients 


f'' + & f' + G&f=0 here () and Cy are constants (100) 


In the following; exarmle a noisy sinusoidal function with an ancular 
frequency of one radian per second is to be srwothed. Vo prediction is 


desired and the crror weighting is of the esmonential type with a 
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smoothing time constant of one second. 


Step 1. gy, = cost (101) 
| go = sin t (102) 
Step 2. M(t) = ee u(=t) (103) 
Step 3. 5° -W5 
Pa. fn (10) 
-1/5 2/5 
i= 10 
Oe 
E(0) = 0 (107) 
Step 6. w(t) = (2 cost = sin +) et u(t) (108) 
Bep7. zs) St - (109) 
s° + 25 +2 


Step 3. Ficure 9 shows the synthesis of this filter as a driving point 


impedancee 





As right have been anticipated, the filter is a resonant circuit. 
At an angular frequency of one radian per second its immedance is purely 
resistive and equal to one ohm. The size of the elements should cause 
no alarm since they depend woon the desired frequency and impedance level 
of the filter. For very low frequencics, it :tight be convenient to use 


the mechanical analogues of the electrical clements. 


ni, 











8/5 henry 


1/2 farad 1/3 ohm : E out 


L/S ohm 


JIVINS 


Figure 9. Filter for Gnoothing Simisoidal Functions 


It appears nossible to construct a filter for recurrent square or 










sairtooth waves by designing them for the fundamental frequency and a 
suitable number of the proper harmonics. A major difficulty with this 
method is that of finding the inverse of a matrix with a large nunber of 


elements. 


5. Polynonial Filters. 

As an examle of the design of polynomial filters, the steps for 
constructing a filter which smooths and predicts a first order poly~ 
nomial or rap will be stomm. The error weighting is of the exmonential 


type with a smootning time constant of one second. 


Brep 1. ey = 1 (110) 


oo =e (111) 


1,0 











Step 2. M(t) = e& u(t) (122) 


2 =) 












Step 3. | 
ya on 
Step ). 2 4 
sJ} = 
Step 5e ge, (9) = 1 (115) 
£5 (8) = 6 (116) 
step 6. w(t) =[(24e)-t4e)] chute) (117) 


See eee we 


(118) 
(s +1)° 


Step is Z(s) 


Step 8. ‘The synthesis of the filter as a driving voint imedance is 


miom in Ficure 10. 


Figure 10. Predictor ior a First Order Polvnorm at 
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Tables 1 and 2 give the inverse matrices for polynomials up to and 
including the fifth order. Table 1 is for rectangular weighting with a 
smoothing time of 1 second while Table 2 is for exmonential weighting 
with a smoothing time constant of 1 second. If T is other than 1 second, 
the tables may still be uscd by scaling the problem properly. for 
example, the expression for the weighting function of a predictor for a 
first order polynomial using rectangular weighting and a smoothing time 


of 1 second is 


W(t) = fut oe - (6+ 12 2 +] fac = oe | (119) 


and the corresponding exwression for a general smoothing time T is 


W(t, T) = ahs 6$ - wo 22) $] fate - «=| (120) 
T e ah 


Figures 11 and 12 show the family of polynomial smoothing filters 
derived from Table 2. Tnese filters are of the low pass type with an 
increasing response to nigher frequencies as the order of the polynomial 
is increased. The eaweiies frequency transfer functions of the nm th 
order polynomial smoothing filters shovm in Figures 11 and 12 have been 
found to talce the forn 


mn 
Cree * 








H(s) = 1 - (121) 
Figure 13 shows the amplitude and phase response versus frequency 
for first order polynomial predictors similar to the one in Figure 10. 
Figure 1 shows the sane information about the polynomial filters in 
Figures ll and 12. A positive increase in prediction time or polynomial 


order evidently tends to increase the noise output of tne filter. 
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Table l. 


Inverse. or fa Mntrices for Polynomials of Order nm. 


a 
ae 


Li(t) 


36 
30 


300 


11:00 
630 


36 
630 
3360 
7560 
7560 
2772 


u(-t) = 


2] 


u(—t-1) 


30 
180 


36 
uo 2 


180 180 


300 
4,800 
18900 
26880 
12600 


630 
11,700 
88200 

211680 
220500 


83260 


Ly 
n= 1 
6 
16 
120 
m & 
20 
1:0 
1050 11,00 
18900 26880 
79380 117600 
117600 =: 179200 
56700 88200 
3360 7560 
88200 211680 
5680 1111200 
U:11200 3628800 
1512000 3969000 
582120 1552320 


43 


Be 


120 


27100 
1680 


630 
12600 
56700 
88 200 


7560 
220500 
1512000 


3969000 


120000 ~ 


171:6360 


2700 
61,80 


1,200 


1,0 
1680 
1200 
2800 


ett2 
83160 
582120 
1552320 
1746360 
6985h) 











Table 1. 
Inverse or cal lintrices for Polynomials of Order n. 


L(t) = u(-t) =~ u(—t-1) 


[a | Pe 





3 


6 12 
9 36 30 16 120 21,0 1,0 
me = 2 36 192 180 120 1200 2700 1680 
30 180 180 til 2h0 2700 8680 200 
U0 1680 200 2800 
25 300 1050 1,00 630 
300 1,800 18900 26880 12600 
n= 1050 18900 79380 117600 56700 
U:00 26880 117600 179200 88200 
630 12600 56700 88200 —-L00 
36 630 3360 7560 7560 e772 
630 1700 88200 211680 220500 83160 
3360 88200 56180 1):1.200 1512000 582120 
> 7560 211680 11:31200 3628800 3969000 1552320 
7560 220500 1512000 3969000 110000 © 176360 
2772 83160 582120 1552320 176360 6985u, 











Table 2. 


Inverse or [4] Matrices for Polynomials of Order n. 


I(t) = o* u(-t) 
" = 
n= 1 

1 1 
SS el L 6 2 6 
2 5 1 6 wm ne yl 
as | 
V2 iL 1/hs : 2 mani Gy 2 | 


YW6 1/2 Wh 1/36 


C 10 5 5/6 12k; 

10 30 35/2 19/6 1/6 

S. 2572 23/2 o/s 1/8 
5/6 19/6 9/\ 17/36 1/36 
1/2h 1/6 1/8 1/36 NO 


6 15 10 15/6 Wh 1/120 

15 55 85/2 23/2 29/2): 1/2h 

10 85/2 73/2 127/12 7/6 1/2h 
15/6 23/2 127/12 13/4 3/8 1/72 
Yh 292k 7/6 3/6 =: 13/208 1/576 
1/120 1/2h 2) 1/72 1/576 «1/00 











Oe 333 0.25 


0.75 





ms 2? n= 3 


Figure ll. Zero Through Third Order Polynomial Smoothing Filters. 


u(t) = e* u(-t) 
All resistances, inductances, and capacitances are given in ohms, henries, 


é 


and farads respectively. 


ho 




















0.2 = 7-656 
3.526 
0.5h7 
a 20057 
— 
1,029 
> 5.689 
es 
0.1667 Le 7: 
Co 2eb8hh 
0.9115 219k 
2 16.25 
2.215 0.903 


8.127 


Figure 12. Fourth and Fifth Order Polynorial Smoothing Filters. 
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6. Filters for a Combination of Functions. 

Since any adcitive or mitiplicative combination of simsoids, 
exponentials, and polynomials is a solution to a reduced equation of 
some linear differential equation with constant coefficients, it may 


He handlieds ‘Slichfanctiong 46 4 «in t, e° Hin t, and + e° 


Sin t my 
thus be treated after the cormlcte set of functions forming a closed 
set upon translation in time is found. A filter wnich is the additive 
conbination of a first order polynomial and a pemeeta can be designed 


as follows. 





Step 1. g1 = cos t (122) 
go = sin a (123) 
g3 =1 (121) 
gq), = + (125) 
Step 22 H(t) = e® u(-t) (126) 
Step 3. 3/5 V5 1/2 0 
=1/5 5 =s/2 @/2 
Ea =— (127) 
Ye 2 1 «ll 
0 1/2 el 2 
Step h. 12 @eh Sa ee 
~); 8 8 2 ‘ 
ae i2 
CA ww, 8 20 ~Ss«#B — 


Lg 








Step 5. (0) = 1 (129) 


g3(0) ae | (131) 
g),(0) =0 (132) 


Step 6. W(t) = |6 ~ 2 = 2cost -  sint| &* u(t) (133) 


3 2 
Step 7e Z(s) = all®_ + 66 + Ce + 2 (13h) 


t 4 hs? + 78% + 68 + 2 


Step 8. Figure 15 illustrates one method of realizing this filter as 


a arriving point impedances 


I in 


Wh farad 
E out 





Oe 






256/35 henry 





126/49 ohn 









49/6 farad 


128/35 ohn 





Figure 15. Smoothing Filter for a Ramp and Sine Wave. 
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From equation (13) 
Z(jl) =1 (135) 
z(0) = 1 (136) 


Thus in the steady state the driving point impedance for a sinusoidal 
input with an angular frequency of one radian per second or for a 
constant input is a pure resistance of one olm. When the inout is 
equal to t, the output may be found by multiplying Z(s) by the Laplace 
transform of t, that is 1/s% The inverse Laplace transform of the 
product has a steady state term equal to t. The steady state output 
divided by the input is then t/t or 1. Although the filter is con= 
structed with reactive elements, its steady state impedance to a 
sinusoid, a constant, and a ramp input appecrs to be a pure resistance 
of one ohm. A phenomenon alin to resonance is thus observed for 


fonctions of time other than sine waves. 


7e Resvonse of Various Polynomial Filters in the Absence of Noise. 
In order to evaluate the polynormal filters, the resvonse of the 

filters to standard input signals in the absence of noise can be 

calculated from equation (5). The standard signals chosen for this 


evaluation are 


Rh =1 (a constant) (137) 
. 1 e +t (a ramp) (138) 
Ry — 1 ts 4° (a quadratic) (139) 
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The graphs on the following pages snow the desired response to these 
signals as dashed lines and the actual resnonse as solid lines. for 
smoothing filters the desired steady state resnonses to Ry» Roy and Ry 
are 1, 1¢ +t, andl+ t+ t; for integrating filters, t, t + t2/2, 
and t + t°/2 + t3/3; and for differentiators, 0, 1, and 1+ 2t respec- 
tively. 

Figures 16 through 21 inclusive illustrate the response of smoothing 
filters designed for constants, ramps, and quadratics with both rectan- 
gular and exponential weighting. It should be noted that the steady 
state response of the rectangular filter . reached abruptly at the end 
of the smoothing period while the exponential filter requires five to 
ten time constants. Filters designed to smooth an m th order polynomial 
will also smooth lower order polynomials. If a polynomial of order 
m+ q*#t il is put into the filter, the output will have an error which is 
a polynomial in t of order q, where q is a positive constant. 

The resvonse of polynomial smoothing filters to a unit step input is 
illustrated in Figures 22 and 23. As the order of the polynomial for 
which the filter is designed increases, the overshoot and frequency of 
transient oscillation is observed to increase. 

Figures 2h, 25, and 26 show the response of filters designed to 
predict a ramp function with four different values of prediction tine 
and three different error memory functionse The influence of the error 
memory function upon the transient response can be clearly seen. 

Responses of filters derived by dividing the complex fremiency trans- 
fer fumctions, H(s), for the constant and ramp smoothing filters by the 
variable s are showm in Figures 27 through 30. These curves indicate that 


a filter which is to integrate volynomials up to the m th order should be 


oe 








based upon polynomial smoothing filters of the m+ p th order, where p is 
the number of integrations to be verformed. This might be anticinated 
from the fact that an integrator increases the order of the outnut poly 
nomlal as compared with the inrut polynomial. 

Differentiators have responses as illuctrated by Figures 31 through 
36. These filters differentiate all polynortials wp to the order m if they 
are based on polynomial snoothing filters of at least the order m-q, 
where q is the number of differentiations. If the rectangular or cmor 
ential error memory fimctions are used, the res’orse curves show discon= 
tinvities in the outnut. These discontinuities indicate that the filters 
have an infinite frequency response and hence would not be very practical 
for eliminating nigh Irequency noise cormonents. Tne discontimiities 
result from the fact tnat the derivative of a wit step fimction is the 
unit impulse or Dirac delta function and may be climinated by choosing 
error memory functions which never have an infinite slope. The responses 


of two filters having such error memory functions are shovm in Figures 35 


and 36. 
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